1. Field of the Invention
The present invention relates to stereo image display apparatuses capable of displaying stereo images to the viewer with left eye and right eye images with a binocular parallax and, more particularly, to improvements in the stereo image display apparatus for alleviating the departure from the natural sense of viewing and fatigue of the viewer viewing stereo image.
2. Discussion of the Related Art
As visual display apparatuses or systems, various stereo image display apparatuses for displaying images viewed as stereo image have been proposed.
FIG. 18 is a perspective view showing a head-mounted display (HMD) 700 as an example of such stereo image display apparatus. The illustrated HMD 700 is a binocular stereo display. The HMD 700 has a frame 702, which is mounted on the viewer's head and supports a left and a right display element and also a left and a right enlarging optical system 701 in front of the viewer's left and right eyes. Thus, a left eye image is displayed to the left eye, while a right eye is displayed to the right eye, whereby the viewer can view the image as a stereo image. The frame 702 has a sensor support 703 supporting a head motion sensor 704, which is located on the head and detects motion of the head. Thus, the viewer can view the image in correspondence to the motion of his or her head. A data processor 720 is connected via a cable 722 to a connector 706, which is supported on a connector support 705 provided on the frame 702. A loudspeaker 709 for outputting sound is provided in each air. The data processor 720 has operating buttons 720a which are operable by the user for various operations. With the usual stereo image display apparatus such as the HMD, the viewing distance and the verging distance fail to coincide with each other, as will be described later in detail, thus resulting in a departure from the natural sense of viewing.
FIGS. 19(a) to 19(c) are views for describing how a left eye image and a right eye image are viewed as a stereo image in the stereo image display apparatus. These figures show an example of a stereo image viewed by the left and right eyes. The image includes two objects, i.e., a sphere and a triangle pyramid, the sphere becoming closer to the viewer. In this case, the left eye and right eye images are changed from those shown in FIG. 19(a) to those shown in FIG. 19(b) and then to those shown in FIG. 19(c). As shown, the sphere is moved toward the center while being gradually increased in size. This means that the binocular parallax is being gradually increased.
FIG. 20 shows the way in which the images shown in FIGS. 19(a) to 19(c) are viewed with the two eyes. Increasing binocular parallax leads to verging for merging, (i.e., reaching or going to reach a viewer's state of perceiving one image on the basis of a plurality of images), so that the viewer's eyeballs are turned inward. This rotation of the eyes is called vergence, and the angle of the rotation is called vergence angle in the illustrated definition. Also, the distance between the optical axes of the eyeballs in vergence and each eye is called parallax distance. In the HMD, the parallax distance is equal to the distance between the point of intersection of the main beans of the left and right images and the main plane of the eyepiece optical system. Thus the vergence of the eyeballs immediately causes accommodation. With increasing vergence angle, the accommodation tends to be closer. Conversely, with reducing the vergence angle, the accommodation tends to be further apart. In the stereo image display, the plane in which image can be viewed with the best contrast is fixed. In the HMD, the distance from this plane to each eyeball is the viewing distance. In this connection, inconsistency has heretofore taken place. Specifically, the above phenomenon occurs not only in the HMD but also in various stereo television sets, such as those of a shutter switching system, a lenticular system, etc. In these systems, the viewing distance of stereo television is the distance from the display surface of the display such as a CRT, to each eyeball of the viewer.
Viewing an image with great verging distance changes as a stereo image in a state that the viewing distance and the verging distance do not coincide, leads to unnatural viewing. This problem may be avoided by producing an image with less fly-out changes. Doing so, however, weakens the impact of image as a stereo image.
To solve this problem, Japanese Patent Publication Heisei 6-85590 proposes an HMD, in which the viewing distance is varied according to the image motion or the like through mechanical driving of the eyepiece lens. Japanese Laid-Open Patent Publication Heisei 3-292093 discloses a method of varying the viewing degree by detecting a point viewed by the viewer and moving the lenses according to depth information at the viewed point. These systems permit one to obtain coincidence of the viewing degree and the verging angle with each other.
Japanese Laid-Open Patent Publication Heisei 7-167633 shows a method of controlling the optimum viewing point, which permits the viewer to perceive the depth world of an object in the broadcast range, by calculating the point from the binocular parallax of image, such as the point is reproduced on the surface of a stereo image display unit or at a specified distance from the surface. As a specific means, a parallax map is calculated from left and right images by using a correlation matching map, and then the mean value of parallax of the entire image or weighted mean parallax of a central part of the image is calculated. Using this mean parallax, a parallax controller controls the horizontal read timing of left and right images to cause parallel movement of the image in the horizontal direction. This method does not require any mechanical drive system, and it is thus possible to prevent size increase.
FIGS. 21(a) to 21(c) are views showing left eye and right eye images displayed in a stereo image display apparatus, which was proposed earlier by the inventor (Japanese Patent Application Heisei 8-28856). Like the case of FIGS. 19(a) to 19(c), two objects, i.e., a sphere and a triangular pyramid, are displayed, the sphere becoming closer to the viewer. In this case, the left eye and right eye images are changed from those shown in FIG. 21(a) to those shown in FIG. 21(b) and then to those shown in FIG. 21(c). In this apparatus, the parallax of the left eye and right eye images is substantially fixed irrespective of the motion of the sphere toward and away from the viewer.
FIG. 22 shows the way of viewing of the images shown in FIG. 21 displayed on an HMD with the two eyes. In this case, the verging distance L with respect to the sphere is unchanged although the image of the ball is increased as the ball becomes closer. The triangular pyramid, on the other hand, is moved apart from the viewer although its size is unchanged. In other words, the distance difference between the triangular pyramid and the sphere is increased as in the prior art case. Nevertheless, the verging distance L with respect to the sphere is substantially fixed.
This is owing to the fact that the person's eyes are not so sensitive with respect to the change in the absolute distance although they are sensitive to changes in the relative distance. Experiments conducted by the inventor prove that the viewer viewing a stereo image of only a single object with changing binocular parallax (background being black), cannot perceive distance changes. However, the sense of stereo arises when objects in different motions are displayed simultaneously. This means that it is difficult to recognize a change in the distance of a single object, although changes in the distance between two objects can be recognized. According to the proposal noted above, with the distance difference between the sphere and the triangular pyramid changing as usual and also the sphere changing in size while the triangular pyramid is not, the viewer perceives it as though the sphere is becoming closer to him or her while the triangular pyramid is not changing its position. It is thus possible to provide images with a stereo sense while holding a substantially constant verging distance with respect to the ball. Preferably, the verging distance L of the sphere in FIG. 22 is made coincident with the viewing distance. More preferably, an eye detector judges whether the viewer is viewing the sphere or the triangular pyramid, and the verging distance of the image being viewed is made substantially constant.
FIG. 23 is a view for explaining the status of merging of a stereo image, which is actually displayed on a left and a right display surface. The relation between the binocular parallax and the verging distance L when viewing a stereo image is now considered. With reference to the figure, when merging is attained, the horizontal positions X1 and X2 of the sphere on the left and right display surface when the sphere is viewed to be at a verging distance L and on a horizontal position -H, are respectively driven as equations (1) and (2). EQU X1={d+(H)}/tan .theta. (1) EQU X2={-d+(-H)}/tan .theta. (2)
In these equations, d is the distance from the mid point between a left and a right lens to each lens (the distance being positive for the right eye and negative for the left eye), and .theta. is the half field angle. The horizontal positions X1 and X2 are prescribed as follows.
FIG. 24 is a view showing how the horizontal positions X1 and X2 in FIG. 23 are normalized. As shown in FIG. 24, the normalization is made by setting the horizontal center value of the display region to "0" and the horizontal length of the display region to "2". Equation (1) can be derived from the fact that the triangle with points A to C in FIG. 23 as the apices of the triangle with origin 0 and points X1 and C on the left display surfaces as the apices are similar to each other. Likewise, equation (2) can be derived from the similarity of the triangle with points D, B and E as the apices and the triangle with the origin 0 and points X2 and E on the right display surface to each other.
Equations (1) and (2) can be rearranged into equation (3). EQU .vertline.X1-X2.vertline.=2d/(L.multidot.tan .theta.) (3)
In equation (3), the left side .vertline.x1-x2.vertline. represents the parallax. Equation (3) shows that the verging distance L when the merging is attained is determined independently of the horizontal position H if the parallax is determined.
The permissible change in the verging distance L, i.e., the permissible change in the parallax, will now be considered. FIG. 25 is a graph showing the correspondence relation between accommodation (i.e., state of focus of the eyes) and vergence. The figure shows the permissible range of the vergence accommodation and the parallax ("O Plus E", Seiri Kogaku 15, December 1985, pp. 103). In the graph, the ordinate is taken for the accommodation (parallax) (D: diopter), and the abscissa is taken for the vergence (vergence angle MW). It will be seen from the graph that the vergence is obtainable in a short period of time so log as its changes are within 4 diopters.
In various display apparatuses, it is usual that the frame or an edge part of the display area enters the visual field of the viewer. However, in the system disclosed in the Japanese Laid-Open Patent Publication Heisei 7-167633 and the other prior art techniques described above, as well as some of the apparatuses which were proposed earlier by the inventor, no particular considerations are given to the influence, which is given to the viewer viewing stereo image by the frame of the display area of display means, i.e., the boundary between the display and non-display areas of the display means.
FIG. 26 is a schematic view for describing the influence given to a stereo image viewer by the display area frames (i.e., display area edges) in a display apparatus having a right eye and a left eye image display area.
With reference to FIG. 26, right eye and left eye LCDs 11R and 11L with right eye and left eye image display areas 11Rd and 11Ld, respectively, are provided for the right and left eyes 10R and 10L. Images on the display areas of the LCDs 11R and 11L, are perceived as images of a right eye and a left eye eyepiece optical system 12R and 12L by the viewer through the right and left eyes 10R and 10L.
On the right eye image display area 11Rd of the right eye LCD 11R, a right side image edge and a left side image edge (i.e., boundaries between display and non-display areas) are formed as right and left edges 11Rrr and 11Rrl, respectively. Likewise, on the left eye image display area 11Ld of the left eye LCD 11L, a right side and a left side image edges (i.e., boundaries between display and non-display areas) are formed as a right edge 11Lrr and a left edge 11Lrl.
In the case of FIG. 26, like the case described before in connection with FIG. 22, an image is assumed which contains two objects, i.e., a sphere and a triangular pyramid, the sphere becoming closer to the viewer. The verging distance L with respect to the sphere is unchanged although the image thereof is increased as the sphere is becoming closer. The triangular pyramid, on the other hand, becomes away from the viewer although its size is unchanged. That is, the distance difference between the triangular pyramid and the sphere is being increased while the verging distance L with respect to the sphere is substantially fixed. In such a state, the positions of right edge images 11ir and 11il, which are merged or verged in a binocular visual field formed by the right and left edges 11Rrr and 11Rrl of the right eye LCD 11R and the right and left edges 11Lrr and 11Lrl of the left eye LCD 11L (i.e., the distance between the image of the sphere and the image of the edge), are fixed as shown.
As described before in connection with FIG. 22, the stereo image display system shown in FIG. 26, utilizes the fact as shown in FIG. 22 that the person's eyes are not so sensitive to detect the absolute distance of an object although they are sensitive to relative distance changes. The system thus permits providing an image with a stereo sense as though the viewer sees the sphere becoming closer to him or her while the position of the triangular pyramid is unchanged, while holding the verging distance L with respect to the sphere substantially constant. However, as described before in connection with FIG. 26, in the system of this type the distance between the image of the sphere and the image of the edge is fixed. Therefore, when the viewer sees both the images of the sphere and the edge in his or her visual field, the inconsistency that the relative positions of the image of the sphere which must be becoming closer to the viewer and the image of the image which is fixed in position becomes unconcealed, thus spoiling the stereo sense of the image, i.e., the sense as though the viewer is actually on the site of the image.
In order to evade the problem described before in connection with FIG. 26, it may be thought to move the positions of the right edge images 11ir and 11il merged (or verged) together in the binocular vidual field (i.e., the distance between the images of the sphere and the edge) increased of fixing these positions.
FIG. 27 is a schematic view showing a case, in which the distances of the right and left edges 11Rrr and 11Rrl of the right eye image display area 11Rd of the right eye LCD 11R and the right and left edges 11Lrr and 11Lrl of the left eye image display area 11Ld of the left eye LCD 11L from one another are variable. Increasing the edge-to-edge distance between the two eyes (i.e., between the edges 11Rrl and 11Lrr) is increased as shown in FIG. 27, gives rise to commonly called field struggling when images produced on the right and left LCDs 11R and 11L are to be verged to produce a stereo image.
In the stereo image display apparatus of this type, usually the parallax concerning a left and a right image, is detected from the correlation between the full frames of the left and right images. However, it is not always efficient data processing to compare the full frames of the left and right images unanimously, that is, irrespective of the images (whether the images are thin or coarse) for obtaining the correlation therebetween. On the other hand, imposing a restriction on the images for obtaining the correlation therebetween, may result in an erroneous judgment.